Skip to content

HIP author skill#438

Open
gjenkins8 wants to merge 3 commits into
helm:mainfrom
gjenkins8:gjenkins/hip_skill
Open

HIP author skill#438
gjenkins8 wants to merge 3 commits into
helm:mainfrom
gjenkins8:gjenkins/hip_skill

Conversation

@gjenkins8

@gjenkins8 gjenkins8 commented May 30, 2026

Copy link
Copy Markdown
Member

First pass at a HIP authoring skill:

  • Aim to refine, scope and investigate an the idea first, and promotes validating the idea in #helm-dev or developer call before creating the full HIP
  • Then a section, by section drafting and refinement process

The HIP template is duplicated and embedded in the skill - enables providing much more detailed guidance for the sections than what is currently in HIP-0001

closes: #436

Signed-off-by: George Jenkins <gvjenkins@gmail.com>
@gjenkins8
gjenkins8 marked this pull request as ready for review May 30, 2026 19:02
@gjenkins8 gjenkins8 changed the title Create a skill to author HIPs HIP author skill May 31, 2026
@gjenkins8
gjenkins8 force-pushed the gjenkins/hip_skill branch from 9dd221f to 13c42a8 Compare June 1, 2026 14:19
Signed-off-by: George Jenkins <gvjenkins@gmail.com>
@gjenkins8
gjenkins8 force-pushed the gjenkins/hip_skill branch from 13c42a8 to 5545c86 Compare June 1, 2026 14:19
@gjenkins8 gjenkins8 mentioned this pull request Jun 1, 2026

@TerryHowe TerryHowe left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@scottrigby scottrigby left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few thoughts:

  1. H4HIP as outlined in https://github.com/helm/community/blob/main/hips/hip-0012.md is missing from this
  2. rather than targeted word length for each section, is it better to have instructions that focus more on the spirit of the goal - such as "keep proposals as simple as possible. do not add unnecessary detail other than what's needed" etc?
  3. meta question - what is this ultimately adding for an LLM above what a copy of https://github.com/helm/community/blob/main/hips/hip-0001.md would provide? If there are missing pieces from hip 001 (eg, H4HIP defined in 0012) should we instead amend hip 0001 with any new/missing/clarifying information?

Signed-off-by: George Jenkins <gjenkins8@bloomberg.net>
@gjenkins8

gjenkins8 commented Jul 19, 2026

Copy link
Copy Markdown
Member Author
  1. H4HIP as outlined in https://github.com/helm/community/blob/main/hips/hip-0012.md is missing from this

H4HIP were HIPs targeting Helm 4's development process. We are beyond that now - we now just have "HIPs" IMHO.

Also to be discussed: the HIP frontmatter actually already has a helm-version: field. Which I think we didn't realize when writing the Helm 4 HIP and proposing H4HIPs. So the H4HIP prefix was actually a little redundant.

Finally, H4HIP was simply a thing we prepended to the HIP's PR title. So we can keep doing that with e.g. H5HIP: , without the skill for writing the HIP actually know/caring I think.


rather than targeted word length for each section, is it better to have instructions that focus more on the spirit of the goal - such as "keep proposals as simple as possible. do not add unnecessary detail other than what's needed" etc?

Given LLMs tendency to write overly verbosely, "as simple as possible" are much more ambiguous. In fact, when I updated the skill to remove the word counts and add such language, the eval reported:

My interpretation

Removing the inline Target ~N words anchors backfired. A concrete number is a cheap, strong constraint the model actually applies as it drafts. Replacing it with "as simple as possible, the counts live in the template, treat them as a ceiling" is a softer signal — and it looks like the runs didn't re-derive the numbers from the template, so sections drifted longer. The prose principle "the range is a guide, not a hard limit" may even have licensed expansion.

That said, an update to the skill to use the word counts as guides (not targets, and not hard ceilings) helped a little in evaluations:

e.g. 746af84#diff-61b24769e609a18d3995dcd65eecd2cc83143d97c8b2faee5c4637b8e6f37069R74


meta question - what is this ultimately adding for an LLM above what a copy of https://github.com/helm/community/blob/main/hips/hip-0001.md would provide? If there are missing pieces from hip 001 (eg, H4HIP defined in 0012) should we instead amend hip 0001 with any new/missing/clarifying information?

The "template" is included inline with the SKILL so it can be enriched with more detailed instructions. So yes, these could (perhaps should) be fed back into the original template. Any instructions should be applicable to both people and LLMs. But initially inline allows the skill off the ground without affecting existing process, it is hard enough to get changes into this repo as is without proposing changes to existing HIPs!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create an Agentic Skill for Creating Helm Improvement Proposals

3 participants